Global Search Methods for Neural Network Training

In many cases the supervised neural network training using a backpropagation based learning rule can be trapped in a local minimum of the error function. These training algorithms are local minimization methods and have no mechanism that allows them to escape the in uence of a local minimum. The existence of local minima is due to the fact that the error function is the superposition of nonlinear activation functions that may have minima at di erent points, which sometimes results in a nonconvex error function. In this work global search methods for feedforward neural network batch training are investigated. These methods are expected to lead to \optimal" or \near-optimal" weight con gurations by allowing the network to escape local minima during training. The paper reviews the fundamentals of simulated annealing, genetic algorithms as well as some recently proposed de ection procedures. Simulations and comparisons are presented.